1. Technical Field
The invention relates generally to object identification and recognition. More particularly, one aspect of the invention is directed to monitoring and characterization of an object in an image, for example an animal or a person, using video analysis.
2. Background Art
Video analysis has developed over the past few decades to become an integral part of machine operations in manufacturing using machine automation. For example, video object recognition and pattern recognition has been used to orient and align various pieces of a product for machining and assembly in various manufacturing industries. One such use is in the manufacturing of semiconductor integrated circuits and microelectronic packaging. In this case, pattern recognition has made great inroads because the size of the work product is microscopic and orientation and alignment of the work product is thus far too tedious for a human being to do consistently and accurately over a large number of pieces.
In recent years, military has carried out research to use video to track moving targets such as tanks and vehicles, in the scene. Other positioning instruments such as global positioning system will be used to assist such tracking.
Another application for video analysis is monitoring animal activity in laboratory testing for the pharmaceutical and biological sciences. One particular area is monitoring animal behavior to determine the effects of various new drugs or gene changes on a particular type of animal. One such animal used in laboratory testing is the mouse.
Over the last two decades, major technological advances have enabled scientists to build a rich repository of mouse models. Model organisms are an important tool for understanding and dissecting human disease and biological process. Because mice and humans share many of the same fundamental biological and behavioral processes, this animal is one of the most significant laboratory models for human disease and studying biological processes in mammals. However, the adequate behavioral characterization (behavioral phenotyping—the impact of a genetic manipulation on visible characteristics of an organism) of genetically engineered mice is becoming a serious bottleneck in the development of animal models; an exponentially increasing number of genotypes are created, but the behavioral phenotyping is often at best rudimentary or is abandoned completely. This is because presently the phenotyping process is largely manual, time consuming, and insensitive to subtle phenotypes.
Video technologies for mouse behavior analysis have been introduced and several products are commercially available. However, these technologies are still primitive and the functionality of the products is far from adequate for the research purposes. There are presently two types of systems available for monitoring mouse behavior, those that identify individual behaviors and those that identify only the location of the mouse.
The most basic state-of-art mouse behavior analysis systems rely on traditional analog technologies that can only treat a mouse as an indivisible object and identify the mouse location. All the information about a mouse is packed as a point in the space and a lot of important information about mouse behavior is lost. The best these systems can do is to find the position of the mouse. Systems like San Diego Instruments' Photobeam and AccuScan Instruments Inc.'s Digiscan Line of Animal Activity Monitoring, Columbus, Ohio uses simple and rudimentary photo-beams to detect and track the positions of mouse. These systems trackers have a very low spatial resolution, limiting their output to a rough measure of the animal's activity. They cannot differentiate even such basic behaviors as locomotion and circling. Adding a time line for the locus of mouse point is all they can offer. Other animal location type systems used to monitor animal motion include those described in U.S. Pat. Nos. 3,100,473; 3,803,571; 3,974,798; 4,337,726; 4,574,734; and 5,816,256.
The other systems in the field are the systems that identify individual behavior using video. The existing video analysis systems (e.g. Noldus Observer/Ethovision, Sterling, Vir.; HVS Image, Hampton, UK; AccuScan Instruments Inc.'s VideoScan2000 System; and San Diego Instruments Poly-Track system, San Diego, Calif.) do not meet expectations either. Digitized images from video are used to capture the body of mouse and provide quantitative data about the position and movements of the animal and the pattern of these variables across time. They do not just treat the animal (e.g., mouse) as a point in the space. Instead, they handle it as a block of pixels. More information is preserved. However, they can only make use of a few simple features. For example, the mass center of the animal (e.g., mouse) is calculated and used as a means for tracking the animal (e.g., a mouse). As such, a lot of information that is critical to identify the animal's behaviors such as different postures, positions of portions of the animal's body such as limbs, is lost. These systems can only distinguish basic behaviors such as locomotion, and cannot automatically identify simple animal postures such as eating, rearing, and jumping, not to mention complex behaviors such as skilled reaching. Such behavior identification requires human intervention and input.
In addition, these systems are often developed for rats that remain relatively stationary in shape as they are in locomotion. However, other animals such as a mouse frequently stretch out, making their center of mass much less stable than a rat. As the center of gravity shifts rapidly and frequently, this falsely adds to measures such as distance traveled, making these systems highly inaccurate for mice. Further, the systems are devised to study white rats on a dark background and are not accurate for tracking other animals such as brown or black mice.
The most advanced systems are those offered by Noldus. The Noldus Observer system has a video camera, TV monitor, a high end VCR, and a PC system, all hooked together. The camera takes video footage of the mouse in a cage. This video is recorded on videotape, digitized, input into the PC system, and displayed on the computer monitor. Although the human observer can control the recorded video that is displayed, the human observer still needs to look at the animal on the screen, decide which behavior the animal is engaged in, and enter (by typing) the information into a mechanism provided by the system for storage and later analysis. While this system facilitates observation of behavior, it does not automate it, and is thus prone to human error and extremely labor intensive. The tasks of coding behavior throughout the day and building a profile of behavior for different types of animals and different strains of the same animal (e.g., different strains of mouse) is prohibitively time consuming with this equipment.